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ABSTRACT 



This study examines how accurately first -year undergraduate 
students at a public university self-report family income. A comparison of 
student -reported income to financial aid records found that less than 40 
percent of students in three incoming classes of first-year undergraduates 
(1995, 1996, 1997) accurately reported family income on an entering student 
survey. Three sources of data were used for the study: first-year student 
self-reported survey data; institutional student data records; and 
institutional financial aid files. Other measures included race and predicted 
grade point average. The results show a significant negative correlation 
between self-reported and actual earnings, with students in the lower income 
categories more likely to over-report actual family earnings and students in 
higher income categories more likely to under-report actual family earnings. 
However, once income level is controlled for, minority students are 
significantly more likely to under-report actual earnings, as are students 
whose parents have not earned at least a baccalaureate degree. Gender was not 
a significant factor in any of the three years measured. (Contains 9 
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The Validity of Students’ Self-Reported Family Incomes 

This study examines the accuracy with which first-year undergraduate students at a public 
institution are able to report family income on an entering student survey administered during a 
summer orientation program for new students. A comparison of student-reported family income 
to financial aid records found that less than 40% of students were accurate in their reporting of 
family income on an entering student survey. A significant negative correlation exists between 
the accuracy measure and actual earnings (r=-.554), with students in the lower income categories 
more likely to over-report their family’s actual earnings and students in the higher income 
categories more likely to under-report actual family earnings. Additionally, once income level is 
controlled for in the model, minority students are significantly more likely to under-report actual 
earnings, as are students whose parents have not earned at least a baccalaureate degree. 
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earnings including: 1) a lack of clarity on the definition of family earnings, 2) faulty memory, 3) 
conscious or unconscious distortion of earnings, and 4) miscoded responses. 

The limited research that has investigated the validity of self-reported family income has 
suggested that adolescents and young adults appear to be aware of both the educational 
attainment and occupational status of their parents, (Borus and Nestel, 1973; Collins, 1973; 
Corcoran, 1980) but appear to be less aware of their parents’ earnings (Kayser and Summers, 
1973). Kayser and Summers (1973) compared youth and parents’ reports of income and found 
that they were statistically unrelated. More recently, Romano and Mareno (1994) compared 
community college students’ reported family incomes to the income reported on the parents' 
federal income tax returns and found that less than one-third of students were able to accurately 
report family income. In a related study of financial aid awards, Trusheim (1994) found that 
college students had little difficulty accurately reporting the type of financial aid awards that they 
received, but were far less able to accurately report the award amounts, usually underestimating 
their awards. 

Accuracy of reporting has also been shown to vary by income level, race, sex, parents’ 
education and job status (Romano and Mareno, 1994; Olivas, 1986; Tusheim, 1994). Previous 
research, focusing on the accuracy of estimates of family income for use in financial aid 
decisions, found that financial aid applications for students from low-income families more 
frequently contained overestimates of family income while applications from higher income 
families tended to underestimate family income (Romano and Moreno, 1994; Sharon and Horch, 
1972). Additionally, because a higher proportion of minority students come from the lower 
income families, these students are more likely to over-report income than are their Caucasian 



classmates (Olivas, 1986). 
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The Validity of Students’ Self-Reported Family Incomes 



Introduction 

Each year a large number of higher education institutions gather information on parents’ 
income via student surveys. Subsequently, these self-reported measures of family income are 
used as a proxy for family socioeconomic status in research studies, institutional profiles, and 
higher education reports. Yet little research has been published that investigates the potential 
response bias introduced when students are asked to report family income. This study examines 
the accuracy with which first-year undergraduate students at a public institution are able to report 
family income on an entering student survey administered during a summer orientation program 
for new students. In addition, various student characteristics are examined to determine if 
particular groups of students are more aware of their family’s income than are others. 

The issue of response bias, particularly as it relates to self-reported personal economic 
indicators, is not new (Perry and Crossley, 1950) but it has had limited impact on the use of such 
measures by higher education researchers. “Despite substantial evidence of the invalidity of 
survey responses, and despite a considerable array of textbook cautions about nonsampling error 
(Bradbum and Sudman, 1988; Fowler 1984), the usual strategy adopted by researchers seems to 
be ‘indifference’ (Jencks et al., 1972).” (Trusheim, 1994). 

An early study of response bias related to personal income measures was conducted by 
Borus (1966). In his study of self-reported earnings for participants in a work program, Borus 
found that discrepancies did exist between respondent- and employer-reported earnings, and that 
respondent characteristics significantly contributed to these discrepancies. Characteristics 
contributing to response bias included: sex, age, education, level of earnings, and motivation to 
answer correctly. Borus also identified several potential causes of response error in self-reported 
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Most higher education studies have looked at the accuracy of family income by focusing 
on the validity of estimates of income as reported on a financial aid application. Only a single 
study was found to examine the accuracy of survey-reported family income (Romano and 
Moreno, 1994). Therefore, this study was designed to contribute to the literature by looking at a 
topic for which little research exists and to investigate the consistency of findings across multiple 
student cohorts. 

Methodology 

The three sources of data used to support this research study included: (1) first-year 
student survey data, (2) institutional student data records, and (3) institutional financial aid data 
files. Data were examined for three separate incoming classes of first-year undergraduate 
students: 1995, 1996, and 1997. Table 1 compares the demographics of the entering first-year 
class for each year with the sample used for this research study. Criteria used for selection into 
the sample included: family income information from the entering-student survey, the father’s 
and mother’s salary and wages and/or parents’ annual gross income as reported on the student’s 
financial aid application, and “dependent status” on the parents’ federal income tax return as 
reported on the financial aid application. 

Table 1. Comparison of Study Sample to Entering Student Population 



All Incoming Freshman Study Sample 





1995 


1996 


1997 


1995 


1996 


1997 


Number of Students 


2,642 


2,808 


2,935 


783 


664 


1,046 


Gender ( % Female) 


59% 


59% 


60% 


63% 


65% 


64% 


Race (% Caucasian) 


86% 


84% 


84% 


81% 


81% 


79% 


Parent’s Education(% with BA Degree) 


61% 


55% 


55% 


48% 


43% 


40% 


Predicted GPA (Average) 


2.50 


2.54 


2.59 


2.54 


2.61 


2.62 


Percent Applying for Financial Aid 


58% 


61% 


63% 
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Survey Income. The self-reported family income measure, as reported on the first-year 
student surveys administered during orientation, asked students to report their parents’ “pre-tax 
income” by selecting from a list of thirteen income ranges. Because these income ranges are not 
of consistent width, the survey categories were recoded into six categories of $10,000 income 
increments each, one category of $15,000 and two categories, with income increments of 
$50,000 and $75,000 respectively. The two categories wider than $15,000, representing the two 
highest income categories, were eliminated from the analyses because the range of incomes 
represented by these categories prohibits making comparisons of the accuracy of student 
responses. Thus, survey income as used in these analyses consists of seven income categories for 
students whose self-reported family incomes ranged from $0 to $75,000. 

Actual earnings. Institutional student financial aid records provided the actual family 
income as reported on the student’s financial aid application. Although this measure is also self- 
reported, applicants are instructed to report both mother’s and father’s wage and salary income as 
reported on specific lines of their parents’ federal income tax documents. Over 91% of the 
sample indicated that the reported income had been obtained from their parents’ actual federal 
income tax retum(s); the remainder used an estimate of income. The decision to use wage and 
salary income rather than adjusted gross income (AGI) was based on the belief that students are 
more likely to be aware of their parents wage and salary income and less aware of other sources 
of income (dividends, interest, farm income, self-employed income, etc.) and losses which are 
included in the AGI calculation. In addition, in analyzing the AGI it was found that these data 
contained many more outliers and were less normally distributed. AGI was used in the few cases 
where no data were reported for parents’ wage and salary income. 
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Accuracy. To determine the degree to which students accurately reported their family’s 



actual earnings, the father’s and mother’s wages and salaries were summed and converted into 



categories matching those of the survey income measure. Once converted to categories, these 
income values were subtracted from their survey income category value, as reported during new 
student orientation. The resulting accuracy measure ranges from -6 to +6 with 0 representing a 
match between the two income categories. This thirteen level accuracy measure is normally 
distributed with approximately 39% of the students accurately reporting their family’s actual 



earnings (Table 2). 



Table 2. Frequencies for Accuracy of Self-reported Family Income 





1995 


1996 


1997 


All 


Under-reported (-6 to -1) 


29% 


34% 


34% 


32% 


Accurately reported (0) 


44% 


38% 


36% 


39% 


Over-reported (1 to 6) 


27% 


28% 


30% 


29% 



Parents’ Education. The educational levels of students’ mothers and fathers were obtained 
from the entering survey and combined into a dummy variable to separate first generation college 
students from students who had one or both parents with at least a baccalaureate degree. 
Approximately 43% of students in the study sample compared to 57% of the entering student 
population had at least one parent with a college degree. 

Other independent measures. The institutional student data files provided the additional 
independent measures of race (Caucasian=l, non-Caucasian=0), gender (female=l, male=0), and 
predicted grade point average (GPA), a measure created by combining SAT or ACT scores and 
high school performance. As indicated in Table 1, the sample contains slightly more women and 
minority students than is represented in the overall first-year student population and the predicted 
GPAs are slightly higher. 
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Analyses were performed using a series of univariate and multivariate statistics to 



determine whether various individual and family factors contributed to the accuracy with which 



students report family income. 



Results 



A significant negative correlation exists between the accuracy measure and actual 
earnings (r=-.554), with students in the lower income categories more likely to over-report their 
actual family earnings and students in the higher income categories more likely to under-report 
actual family earnings. Table 3 provides income information for each of the thirteen accuracy 
categories to illustrate the strong relationship between family income and the accuracy with 
which students report that income. Looking at median income, for example, the income values 
for the under reporters, those in the minus category of accuracy, are much higher than the income 
values for the over reporters, those with positive values on the accuracy measure. 



Table 3. Family Income Means and Medians by Level of Reporting Accuracy 



Accuracy a 


N 


Median 


Mean 


Std. Dev. 


Min. 


Max. 


-6 


2 


$ 68,751 


$68,751 


$ 562 


$ 68,352 


$ 69,150 


-5 


10 


72,509 


71,751 


15,535 


51,131 


92,586 


-4 


35 


77,079 


72,424 


17,300 


41,658 


111,507 


-3 


90 


64,509 


70,728 


24,473 


36,919 


182,482 


-2 


183 


61,580 


63,814 


19,692 


21,150 


134,591 


-1 


502 


55,025 


56,218 


21,678 


10,456 


149,388 


0 


969 


36,839 


38,469 


19,829 


0 


74,972 


1 


412 


32,420 


32,222 


16,511 


0 


59,899 


2 


137 


27,440 


26,921 


14,770 


0 


49,583 


3 


84 


17,215 


18,497 


12,363 


0 


54,613 


4 


42 


16,621 


15,926 


9,669 


0 


29,797 


5 


15 


13,503 


12,781 


5,388 


1,974 


19,935 


6 


11 


6,389 


17,866 


6,839 


0 


88,935 



a 0=accurate reporting; negative numbers=under-reporting; positive numbers=over-reporting 



Bivariate analyses were preformed to understand the relationship between the dependent 



measure of accuracy and each background measure: gender, race, predicted GPA, and parents’ 



education. Initial analyses revealed two significant t-tests between subgroups of students on the 
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accuracy measure in the 1996 sample— between females and males and between Caucasians and 
minority students. In addition, a significant interaction was discovered when the accuracy 
measure was analyzed by race and year. In the 1995 sample, both Caucasian and minority 
students had a negative mean accuracy measure indicating that on average, each was under- 
reporting their family’s actual income. For the 1996 and 1997 samples, Caucasians, on average, 
under-reported their family income, while minority students tended to over-report family income. 

Least square stepwise hierarchical regression analyses were performed for each of the 
three years under study and for the three years combined. The dependent variable in the model 
was the accuracy measure, the independent measure was actual earnings for the student’s family, 
and background variables included gender, race, predicted GPA, and parents’ educational level. 

A comparison of the regressions for each of the three years revealed little substantive differences 
in the outcomes. Therefore, in the interest of parsimony, only one analysis will be discussed — 
the regression model for the three years combined. 

Table 4. Stepwise Hierarchical Regression Analysis to Determine the Accuracy of 

Student-reported Family Income 



Variable 


B 


SEB 


P 


Model 1 








Independent 








Actual Earnings 


-.000“ 


1.11 


-.55*** 


F Statistic 






1073.47*** 


Adjusted R 2 






.31 


Model 2 








Step 1 -Independent 








Actual Earnings 


-.000 b 


1.15 


-.59*** 


Step 2-Backeround variables 








Female 


-.142 


0.05 


-.04* 


Caucasian 


.327 


0.07 


.08*** 


Predicted Grade Point Average 


-.046 


0.08 


-.01 


Parents’ Education 


-.267 


0.05 


-.08*** 


Change in F Statistic 






15.57*** 


Adjusted R 2 






.32 



*p< 05, **p<.01, ***p< 001 ‘ = -.0000365 b =-,0000388 
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The negative correlation between accuracy and actual earnings indicates that students 
who under-report their actual family earnings are more likely to be in the higher income 
categories. However, once income level is controlled for in the model, minority students are 
significantly more likely to under-report actual earnings, as are students whose parents have not 
earned at least a baccalaureate degree. Therefore, although these students tend to over-report 
when all income ranges are considered, they tend to under-report when looking within a single 
income range. While women also are significantly more likely to under-report family income 
than are their male classmates, caution must be used in interpreting this result because gender 
was not a significant predictor in any of the three individual years. The significance, in this case, 
may be a manifestation of sample size. When the independent measure, actual earnings, is 
regressed upon the accuracy measure, the model explains approximately 3 1% of the variance 
(r=.554); once the background variables are added to the model the amount of variance explained 
rises to 32% (r=.569), a significant change in the F-Statistic. 

Conclusions 

The negative correlation between actual earnings and the accuracy measure is consistent 
with previous studies (Olivas, 1986; Romano and Mareno, 1994) and suggests that student- 
reported family income, as gathered through student surveys, has limited validity for institutional 
decision making. Less than 40% of the students in this study were able to accurately report their 
family’s income on the entering student survey. Unfortunately, this study does not enable the 
identification of the potential causes for response bias. It is difficult to know the degree to which 
parents actually talk with their children about family income matters, but it would appear that 
these conversations are more likely to occur in families with moderate incomes. At the same 
time, this finding may be more related to upper and lower income students’ hesitancy to 
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accurately report their family income when completing a survey in a classroom setting. As 
observed by Borus (1966), part of the response bias may be a result of student’s lack of 
understanding of what is meant by “parents’ pre-tax income” as requested on the survey or 
merely the result of a misplaced pencil lead. 

While this study has the advantage of looking at three different entering classes of new 
students, it still suffers from some of the same limitations as previous studies. First, it is limited 
to a single institution, thereby restricting its generalizability. Second, it is limited to students who 
apply for financial aid, reducing the ability to examine the accuracy of student-reported family 
earnings for higher income families. Finally, it is limited by the quality of the survey income 
measure. 

Implications 

The fact that lower income students tend to over-report family income and students from 
higher income levels tend to under-report, has the effect of constricting the survey income 
measure. While this study was limited to students who applied for financial aid and who reported 
family income below $75,000 on the entering student survey, the findings suggest that 
institutions should be cautious in their use of income information gathered from students in this 
manner. The proportion of students who were accurate in their assessment of their family’s 
income is slightly higher than most other studies (Olivas, 1994; Romano and Mareno, 1994), yet 
still less than 40% were able to accurately determine the appropriate survey income category. 

Additionally, because the students who have greater financial need, particularly those 
from under-represented groups (minorities, first generation college students) tend to overestimate 
their family’s earnings, the potential is great for administrators to underestimate the financial 
needs of their students. This could result in the under-appropriation of financial resources for 
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student support. At the same time, those from higher incomes tend to underestimate their family 
earnings, which could cause administrators to believe that they have many more students in the 
mid-range of incomes. The ramifications are exacerbated when these survey income data are 
reported to governing boards and governmental agencies that aggregate the data across 
institutions to make policy decisions regarding both funding and access. 

Although many institutions rely on student-reported family income measures in their 
research efforts, these findings suggest that the validity of these measures are highly suspect. 
Furthermore, the significance of the background measures to further explain the discrepancies 
between actual earnings and survey income suggests that investigators need to carefully examine 
institutional data before analyzing. Institutional researchers must use caution in reporting self- 
reported family income and in using such measures in research studies. 

Finally, researchers would be well advised to consider the methods for gathering such 
information. For example, the freshman orientation surveys used for this study were 
administered in a classroom setting. The possibility exists that particular groups of students may 
have over- or under-reported their family income not out of a lack of knowledge about their 
fami ly’s financial situation but because they were concerned that their peers sitting beside them 
might observe their responses. Discrimination based on family socioeconomic status must not be 
taken lightly given the importance of peer acceptance for college freshmen. Other impressions, 
such as the student’s perception of his or her status as viewed by the survey administrators, might 
also have led to inaccurate reporting. Finally, given the variations in reporting patterns for 
various subpopulations, it is paramount that institutional researchers disaggregate data so as not 
to draw erroneous conclusions as a result of masked effects. 



ERIC 



14 



FAMILY INCOME 13 



Institutions will continue to use family income information as a means to understand their 
students and as a decision making tool. Therefore, institutional researchers would be well 
advised to conduct similar studies on their students to determine the degree of confidence they 
can place in such measures of self-reported family income. At the very least, institutional 
researchers must consider how they collect, interpret, and share family income information to 
ensure that it most accurately reflects the profile(s) of their students. 
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